ON ACOUSTICS MADRID , 2 - 7 SEPTEMBER 2007 Development of Selectable Viewpoint and Listening Point System for Musical Performance

This paper describes the development of a new visual and audio display on which users can view images and listen to sounds at arbitrary positions. In this system, the ray-space method is used for the selectable viewpoint, and a signal separation method based on frequency-domain independent component analysis is used for the selectable listening point. For the selectable listening point, individual musical instrument performances are extracted from the signals composed by various instruments, and users can listen at any arbitrary positions by synthesizing the signals generated by convoluting the separated signals and head-related transfer functions. We use actual musical performance instead of transduced signals from loudspeakers. In this case, separation performances are influenced by room reverberation, directivity of the musical instrument, and so on. Moreover, since evaluating these separation performances is difficult because we cannot obtain reference signals, we propose a microphone pair selection-type method and a novel evaluation measure of separation performance. We collected the images and the sound signals in a real environment and evaluated the separation performances. As a result, the proposed methods were found to be effective for actual musical performance.